'\"macro stdmacro
.\"
.\" Copyright (c) 2019 Miroslav Foltýn.  All Rights Reserved.
.\" Copyright (c) 2019 Red Hat.
.\"
.\" This program is free software; you can redistribute it and/or modify it
.\" under the terms of the GNU General Public License as published by the
.\" Free Software Foundation; either version 2 of the License, or (at your
.\" option) any later version.
.\"
.\" This program is distributed in the hope that it will be useful, but
.\" WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
.\" or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
.\" for more details.
.\"
.TH PMDASTATSD 1 "PCP" "Performance Co-Pilot"
.SH NAME
\f3pmdastatsd\f1 \- \f3StatsD\f1 protocol performance metric domain agent (PMDA)
.SH SYNOPSIS
\f3$PCP_PMDAS_DIR/statsd/pmdastatsd\f1
[\f3\-D\f1 \f2debug\f1]
[\f3\-Z\f1 \f2maximum udp packet size\f1]
[\f3\-P\f1 \f2port\f1]
[\f3\-v\f1]
[\f3\-g\f1]
[\f3\-o\f1 \f2debug output filename\f1]
[\f3\-s\f1]
[\f3\-r\f1 \f2parser type\f1]
[\f3\-a\f1 \f2port\f1]
[\f3\-z\f1 \f2maximum of unprocessed packets\f1]
.SH DESCRIPTION
.B StatsD
is simple, text-based UDP protocol for receiving monitoring data of applications
in architecture client-server.
.B pmdastatsd
is an Performance Metrics Domain Agent that collects
.B StatsD
data, aggregates them and makes them available to any Performance Co-Pilot client,
which is ideal for easily tracking stats in your application.
.PP
The
.B statsd
PMDA supports Counter, Gauge and Duration (with instances for minimum,
maximum, median, average, 90th percentile, 95th percentile, 99th
percentile, count and standard deviation) metric types, offers multiple
parsing options:
.B Ragel
or
.BR "handwritten/custom parser",
offers multiple aggregating options for duration metric type:
.B "basic histogram"
or
.BR "HDR histogram" ,
supports custom form of
.BR labels ,
.BR logging ,
exports multiple metrics about itself and may be configured either with
an ini file or command line options.
.SH CONFIGURATION
A brief description of the
.B pmdastatsd
command line options follows:
.TP 5
.B \-Z, \-\-max\-udp=<value>
Maximum allowed packet size, any larger then this will be thrown away.
Default:
.I 1472
.TP
.B \-P, \-\-port=<value>
Which port agent will listen to for incoming traffic.
Default:
.I 8125
.TP
.B \-v, \-\-verbose
Verbosity level.
Prints info about agent execution into logfile. Valid values are 0-2. 0 = Default value, shows config information, read socket state, and
first 100 dropped messages. 1 = Shows PMNS and related information. 2 = Most detailed verbosity level, also shows dropped messages above 100.
All levels include those below.
.TP
.B \-o, \-\-debug\-output\-filename=<value>
You can send USR1 signal that 'asks' agent to output basic information about all
aggregated metric into a $PCP_LOG_DIR/pmcd/statsd_{name} file.
Default:
.I 0
.TP
.B \-s, \-\-version
Flag controlling whether or not to log current agent version on start.
Default:
.I 0
.TP
.B \-p, \-\-parser\-type=<value>
Flag specifying which parser to use for incoming packets; Basic =
.IR 0 ,
Ragel =
.IR 1.
Ragel parser includes better logging when verbose =
.IR 2.
Default:
.I 0
.TP
.B \-a, \-\-duration\-aggregation\-type=<value>
Flag specifying which aggregation scheme to use for duration metrics;
basic histogram =
.IR 0 ,
HDR histogram =
.IR 1 .
Default:
.I 1
.TP
.B \-z, \-max\-unprocessed\-packets=<value>
Maximum size of packet queue that the agent will save in memory.
There are 2 queues: one for packets that are waiting to be parsed and
one for parsed packets before they are aggregated.
Default:
.I 2048
.PP
The agent also looks for a
.I pmdastatsd.ini
file in the
.B $PCP_PMDAS_DIR/statsd
directory.
There, the same options may be specified, albeit with slightly different
names as follows:
.RS 5
.P
.B max_udp_packet_size=<value>
.br
.B port=<value>
.br
.B verbose=<value>
.br
.B debug=<value>
.br
.B debug_output_filename=<value>
.br
.B version=<value>
.br
.B parser_type=<value>
.br
.B duration_aggregation_type=<value>
.br
.B max_unprocessed_packets=<value>
.RE
.P
Should an option be specified in both
.I pmdastatsd.ini
and command line, then the latter takes precedence.
Most of the time you will want to configure the agent with an ini file,
as the agent should never be executed directly.
.P
Location of the log file.
By default, a log file named
.I statsd.log
is written in the current directory of
.BR pmcd (1)
when
.B pmdastatsd
is started, i.e.
.BR $PCP_LOG_DIR/pmcd .
If the log file cannot
be created or is not writable, output is written to standard error
instead.
.SH INSTALLATION
If you want to install the
.BR pmdastatsd ,
do the following as root:
.PP
.ft CR
.nf
.in +0.5i
# cd $PCP_PMDAS_DIR/statsd
# ./Install
.in
.fi
.ft 1
.PP
To uninstall, do the following as root:
.PP
.ft CR
.nf
.in +0.5i
# cd $PCP_PMDAS_DIR/statsd
# ./Remove
.in
.fi
.ft 1
.PP
.B pmdastatsd
is launched by
.BR pmcd (1)
and should never be executed directly.
The Install and Remove scripts notify
.BR pmcd (1)
when the agent is installed or removed.
.SH USAGE
Once started,
.B pmdastatsd
will listen on specified
.I port
for any content in a form of:
.RS 4
.P
.B <metricname>:<value>|<type>
.RE
.P
There may be multiple such messages in single datagram, split by a newline character, so this:
.RS 4
.P
.B <metricname>:<value>|<type>\[rs]n<metricname>:<value>|<type>
.RE
.P
is valid as well.
Constraints for each of the above variables are:
.RS 4
.P
.B <metricname> = [a-z][a-zA-Z0-9_.]*
.br
.B <value>      = described further in each metric type subsection
.br
.B <type>       = One of the following: "c", "g" or "ms"
.RE
.P
If
.I verbose
logging is turned on, agent will log every message parsed and related failures.
.P
All recorded metrics will, if parsed and aggregated successfully, be made available under
.B statsd.*
namespace.
.SS 1 Counter metric
Stores metrics as simple counters, adding any incoming values to already existing ones.
.RS 4
.P
.B <metricname>:<value>|c
.RE
.P
Where
.BI value
is positive number.
.P
.B Example:
.P
After aggregating following messages:
.RS 4
.P
.B metric:20|c
.br
.B metric:10|c
.br
.B metric:3.3|c
.RE
.P
Value available to PCP will be:
.PP
.RS 4
.ft CR
.nf
# pminfo \-\-fetch statsd.metric
.P
    inst [0 or '/'] value 33.3
.fi
.ft 1
.RE
.SS 2 Gauge metric
Stores metrics as modifiable values, with an option to either set,
increment or decrement values.
.RS 4
.P
.B <metricname>:<value>|g
.RE
.P
Where
.BI value
can be in a form of:
.RS 4
.P
.BR '\-{value}' ,
when negative value is supplied agent will subtract value stored
with the value passed

.BR '+{value}' ,
when positive value with a leading plus sign is supplied, the agent
will add the passed value to the value stored

.BR '{value}' ,
when a value without any leading sign is supplied, the agent will
set the metric to the passed value.
.RE
.P
Initial value for metric of gauge type is 0.
.P
.B Example:
.P
After aggregating following messages:
.RS 4
.P
.B metric:20|g
.br
.B metric:+10|g
.br
.B metric:-3.3|g
.RE
.P
Value available to PCP will be:
.PP
.RS 4
.ft CR
.nf
# pminfo \-\-fetch statsd.metric
.P
    inst [0 or '/'] value 26.7
.fi
.ft 1
.RE
.SS 3 Duration metric
Aggregates values either via HDR histogram or simply stores all values and then calculates instances from all values received.
.RS 4
.P
.B <metricname>:<value>|ms
.RE
.P
Where
.BI value
is a positive number.
.P
.B Example:
.P
With larger message count, the values may vary based on selected duration aggregation scheme.
.P
After aggregating following messages:
.RS 4
.P
.B metric:10|ms
.br
.B metric:20|ms
.RE
.P
Values available to PCP will be:
.PP
.RS 4
.ft CR
.nf
# pminfo \-\-fetch statsd.metric
.P
    inst[0 or '/min'] value 10
    inst[1 or '/max'] value 20
    inst[2 or '/median'] value 10
    inst[3 or '/average'] value 15
    inst[4 or '/percentile90'] value 20
    inst[5 or '/percentile95'] value 20
    inst[6 or '/percentile99'] value 20
    inst[7 or '/count'] value 2
    inst[8 or '/std_deviation'] value 5
.fi
.ft 1
.RE
.P
.B Note:
.P
Once you send given
.I metricname
with specified
.IR type ,
the agent will no longer aggregate any messages with same.
.I metricname
but different
.I type
and will throw them away.
.SS 4 Labels
StatsD datagrams may also contain key:value pairs separated by commas like so:
.RS 4
.P
.B metric,tagX=X,tagW=W:5|c
.P
OR
.P
.B metric:5|ms|#tagX:X,tagW:W
.RE
.P
Where
.BI tagX
is a
.IR key ,
.BI X
is a
.I value
and
.BI tagW
is a
.IR key ,
.BI W
is a
.IR value .
.P
Both
.I key
and
.I value
of such a pair are
.BR "[a\-ZA\-Z0\-9_.]{1,}" .
.P
Both formats are interchangeable and you may combine them together.
When
.I key
is not unique, right most
.I value
takes precedence.
This is valid:
.RS 4
.P
.B metric,tagX=1:5|c|#tagX:2
.RE
.P
Pair with
.I key
.I tagX
will have value of 2.
.P
You may use these labels to map specific values to some PCP instances.
PCP labels are also assigned to these PCP instances.
Pairs are ordered by
.I key
in resulting instance name and label descriptor.
.P
Single label:
.RS 4
.P
.B metric,tagX=X:5|c
.RE
.P
Such a payload would map to PCP as follows (non-related labels were omitted):
.PP
.RS 4
.ft CR
.nf
# pminfo \-\-fetch \-\-labels statsd.metric
.P
    inst [0 or '/tagX=X'] value 5
    inst [0 or '/tagX=X'] labels {'tagX':'X'}
.fi
.ft 1
.RE
.P
As shown earlier you may also send payload with multiple labels.
When multiple labels are supplied they are split in instance name by '::'.
Example:
.RS 4
.P
.B metric,tagX=X,tagW=W:5|c
.RE
.P
This resolves to:
.PP
.RS 4
.ft CR
.nf
# pminfo \-\-fetch \-\-labels statsd.metric
.P
    inst [0 or '/tagX=X::tagW=W'] value 5
    inst [0 or '/tagX=X::tagW=W'] labels {'tagX':'X','tagW':'W'}
.fi
.ft 1
.RE
.P
.B Note:
.P
Be mindful of the fact that duration metric type already maps to
instances even without any labels.
Sending labeled value to such a metric creates another 9 (as there
are that many hardcoded) instances.
.P
Example:
.RS 4
.P
.B metric:200|ms
.br
.B metric:100|ms
.br
.B metric,target=cpu2:10|ms
.br
.B metric,target=cpu2:100|ms
.br
.B metric,target=cpu2:1000|ms
.RE
.P
Creates 18 instances.
Duration data type and label name compose instance name in following manner:
.PP
.RS 4
.ft CR
.nf
# pminfo \-\-fetch \-\-labels statsd.metric
.P
    ...
    inst [10 or '/max::target=cpu0'] value 1000
    inst [10 or '/max::target=cpu0'] labels {'target':'cpu0'}
    ...
.fi
.ft 1
.RE
.SS 5 Hardcoded stats
Agent also exports metrics about itself:
.TP 5
.B statsd.pmda.received
Number of datagrams that the agent has received
.TP
.B statsd.pmda.parsed
Number of datagrams that were successfully parsed
.TP
.B statsd.pmda.dropped
Number of datagrams that were dropped
.TP
.B statsd.pmda.aggregated
Number of datagrams that were aggregated
.TP
.B statsd.pmda.metrics_tracked
This metric has 3 instances.
.B counter
- Number of tracked counter metrics.
.B gauge
- Number of tracked gauge metrics.
.B duration
- Number of tracked duration metrics.
.B total
- Number of tracked metrics total.
.TP
.B statsd.pmda.time_spent_parsing
Total time in microseconds spent parsing metrics. Includes time spent parsing a datagram and failing midway.
.TP
.B statsd.pmda.time_spent_aggregating
Total time in microseconds spent aggregating metrics. Includes time spent aggregating a metric and failing midway.
.TP
.B statsd.pmda.settings.max_udp_packet_size
Maximum UDP packet size
.TP
.B statsd.pmda.settings.max_unprocessed_packets
Maximum size of unprocessed packets Q
.TP
.B statsd.pmda.settings.verbose
Verbosity flag
.TP
.B statsd.pmda.settings.debug_output_filename
Debug output filename
.TP
.B statsd.pmda.settings.port
Port that is listened to
.TP
.B statsd.pmda.settings.parser_type
Used parser type
.TP
.B statsd.pmda.settings.duration_aggregation_type
Used duration aggregation type
.P
These names are blacklisted for user usage.
No messages with these names will processed.
While not yet reserved, the whole
.B statsd.pmda.*
namespace is not recommended to use for user metrics.
.SH FILES
.PD 0
.TP 10
.B $PCP_PMCDCONF_PATH
command line options used to launch
.B pmdastatsd
.TP 10
.B $PCP_PMDAS_DIR/statsd/Install
installation script for the
.B pmdastatsd
agent
.TP 10
.B $PCP_PMDAS_DIR/statsd/Remove
undo installation script for the
.B pmdastatsd
agent
.TP 10
.B $PCP_LOG_DIR/pmcd/statsd.log
default log file for error messages and other information from
.B pmdastatsd
.PD
.SH "PCP ENVIRONMENT"
Environment variables with the prefix
.B PCP_
are used to parameterize the file and directory names
used by PCP.
On each installation, the file
.I /etc/pcp.conf
contains the local values for these variables.
The
.B $PCP_CONF
variable may be used to specify an alternative
configuration file,
as described in
.BR pcp.conf (5).
.SH DEBUGGING OPTIONS
The
.B \-D
or
.B \-\-debug
option enables the output of additional diagnostics on
.I stderr
to help triage problems, although the information is sometimes cryptic and
primarily intended to provide guidance for developers rather end-users.
.I debug
is a comma separated list of debugging options; use
.BR pmdbg (1)
with the
.B \-l
option to obtain
a list of the available debugging options and their meaning.
.SH SEE ALSO
.BR PCPIntro (1),
.BR pmcd (1),
.BR pcp.conf (5)
and
.BR pcp.env (5).

.\" control lines for scripts/man-spell
.\" +ok+ duration_aggregation_type max_unprocessed_packets
.\" +ok+ time_spent_aggregating debug_output_filename max_udp_packet_size
.\" +ok+ time_spent_parsing metrics_tracked std_deviation parser_type
.\" +ok+ metricname pmdastatsd Hardcoded
.\" +ok+ logfile StatsD statsd Ragel
.\" +ok+ tagW tagX HDR USR udp
.\" +ok+ statsd_ {from PCP_LOG_DIR/pmcd/statsd_<lb>name<rb}
.\" +ok+ ZA zA {both from regex's}
